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FOREWORD 



h portion of the Army Research Institute for the Behaviorai and 
Social Sciences (aRi) program consists of research and developnient in 
areas of educational technology with applicability to military training. 
One research area of interest is the use of altGrnative training media 
for students of varying abilities and characteristics, pevelopment and 
implementation of such syBtems can potentially increase the efficiGncy 
and effectiveness of the training process because the instruction can bo 
adapted to the individual student to a greater extent than would oth©r^ 
wise be possible. 

h prerequisite to the development of aiicb systems is the identifi^ 
cation of those student characteristics and abilities that are predictive 
of training success with specific media. Previous research had saggested 
that student attitudes toward the specific medium involved were potential 
predictors, and in addition had produced the neceasary instruments to 
measure student attitudes toward one medium ? Coniputer^Assisted Instruction 
(CAI) . The purpose of this research was to investigate more cKtensively 
the relationships between student attitudes toward Chi and their perfor^ 
mance while they were receiving instruction ^y . 

Whereas this effort is a basic research study, it initially evolved 
because of and in support of an operational evaluation. The impetus for 
this project was a request frorn the U,S- himy Ordnance Canter and School 
(USAOCfiS) for scales on which to measure student attitudes toward CAI to 
be used in an evaluation of the training effectiveness of the PLATO IV 
CAl systeni. To meet this request, basic research in this area was needed. 
A cooparative effort followed, with USAOC&S receiving the scales for 
their use# and ARI receiving personnel and student support for the first 
phase of the basic research project. This effort, which began during 
FY 75 1 was part of the Unit Training and Educational Technology Systems 
Technical Area work program. Following a raorganization/ the effort 
continued in FY 76 as part of the Iducational Technology and Training 
Simulation Technical Area; specifically the Educational Concepts and 
Evaluation Work Unit area. The entire research effort is responsive to 
FY 75 requirements of Project 2T161101A9ia , "Independent Laboratory In-- 
Houoe Kesearch." 
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Tfeminical Director (Designate) 



5- 



BBLATIONSHIPS BBl^EEN STUDENT ATTITUDES TOWARD COMPUTER- ASSISTED 
INSTRUCTION AND TRAINING PERFORMANCE 



BRIEF 



Requirement I 

To investigate the relationships between military student attitudes 
toward Computar-'Assisted instruction (CAI) and performance whila students 
Were receiving instruction by CAI^ 

Procedure i 

The attitudes of 90 military enlisted personnel toward CAI were 
rneasured before and after they received a 107-minute unit of instruction 
on human audition presented by CAI. Both the attitude pretest and the 
attitude posttest were administered by computer. The instructional mate-' 
rial was divided into four lesson segrnents^ each followed by a review 
(test) sagnient. Students who failed to reach a specified score on any 
review segment repeated the cDrresponding lesson segment. A final test, 
covering the material from all segments # was administered at the end of 
the last segment. Five measures of student performance during the course 
of the instruction were usedi (a) the total score on the final test 
(total score) i (b) the number of errors made during the lesson and review 
segments (errors) i (c) the percentage of responses made during the lesson 
and review segments that were incorrect (percent errors) ; (d) the time 
required to complete the lesson and review segments (time) ; and (e) the 
number of review segments failed (review failures) . 

Findings i 

Scoras on the attitude pretest were not related to scores on any of 
the lesson performance criteria. A significant relationship was obtained 
between the attitude posttest percent errors , indicating that those stu^ 
dents who made a lower percentage of errors during the lesson also had 
more positive end-of^lesson attitudes toward CAI. Change in attitude was 
related to totaL score, errors, and percent errors. Positive change in 
attitude was also related to "good" performance on these criterion mea-^ 
sures , Scores on both attitude scales were independent of student abil-^ 
ity , as measured by scores on the AFQT and the ACS* 



e 



These findings indicate that student attitudas toward Chi are of 
little ov no value in the selection of training media and methods for 
individual students, at least when thci students have had limited prior 
ajcposure to Chi. 



Utilisation of Findings? 

The on-line versions of the attitude scales are available for use 
other organizations and will be used by ARI for future research. 
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RELATIONSHIPS BETWEEN STUDENT ATTITUDES TOmRD COMPUTER-ASS ISTKD 
INSTRUCTION AND TRAINING PERFORMANCE 



INTRODUCTION . 

A central theme in the field of educational technology has been the 
development and evaluation of Tnethods that allow instruction to be indi^ 
vidualized. Both training specialists and educational theorists imooq- 
nizB the importance of adapting training to the individual student if 
significant advances in training efficiency or effectiveness are to be 
inade . 

Training can be individualized in many ways. Self-pacingi. either 
through Programed Instruction (FI) lessons or the self-pacing of entire 
courses, represents perhaps the simplest form of tndividualiaaa training. 
Computer-Assisted Instruction (CAI) lessons that modify the aitiount and 
gequenee of instructional material on the basis of student pe-r£orinance 

(or student choice) represent another way to individualize training. A 
rnore compleK method is to assign each student to a particular training 
method or medium that is expected to be most effective for him or her. 
The "bast" method may well differ for each student as a fmction of the 
type of instructional material presented. There are three wlnimal 
requirements for implementing individualized training of the last types 

(a) multiple methods and media by which students can accomplish the same 
training objectives i (b) decision models for selecting the best training 
methods and media for individual students; and Cc) mechanisms for imple^ 
menting those decision models. Requirements (a) and (c) are v?all within 
the scope of today's technology. However, at present there is inauffi- 
cient knowledge regarding the interactions among student characteristics, 
instructional treatments (methods and media) , and training content to 
permit implementation of such an approach to individualized training. 

Student attitudes toward training methods and media constitute one 
possible basis for media selection decisions. The relatively high im- 
portance of such attitudes, particularly toward CAI, PI, and Computer-* 
Managed instruction (CMI) , as perceived by both Army trainers and 
researchers, made them a logical topic for further investigation. Army 
trainers typically assess student attitudes toward CAI or CMI systems as 
a part of the evaluations of those systems. The evaluations of the PLATO 
IV CAI system at the Army Ordnance Center and School and the Computerized 
Training Systems Project ABACUS at the Army Signal School are recent eK- 
amples. High perceived importance by researchers is evidenced by the 
substantial nunter of studies that have collected student attitude data. 
King (1975), in a review of the research literature in this area, cited 
44 such studies. 
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However, this importance has produGecl neither well-integrated nor 
consistent results, particularly with regard to relationships between 
student attitudes and student perfornianca in the instructional situation. 
King (1975) notad two methodological problenis which, it would appear, are 
at least partially responsible for this lack of integration and conBis- 
tency. First, although student attitude data are frequently collected (as 
noted above) /there has been no effort to investigate student attitudes in 
any integrated fashion* Only rarely are student attitudes the primary fo- 
cus of the research. investigators are justifiably more concerned with 
the effects of the independent variables on cognitive or psychomotor per- 
formance than with effects on student attitudes. Second, there is little 
agreement on an operational definition of student attitude. There have 
been some attempts to achieye consistency in the use of measuring instru^ 
ments^ five of the studies cited by King^ and one additional study to be 
cited here (Gallagher, 1970) , have used either a scale developed by Brown 
(1966) or modifications thereof. For the most part, however, the scales 
used are designed ad hoc with neither the items used nor the metric prop- 
erties of the scales described. As King noted, *'Most studies use 
eKperimenter-constructed tests which have unknown or unrGported 
reliabilities (1975, p,7)." 

Relatively few studies have attempted to investigate what, if any, 
relationships exist between attitudes toward PI, CAI, or CMI and perfor- 
mance in a course of instruction taught by one of those methods. Studies 
that have attempted to find relationships have produced mixed results* 
Doty and Doty (1964) found posttest attitude (attitude measured following 
the instruction) toward PI to be positively correlated with lesson 
achievement for female students but not for male students* Gallagher 
(1970) found no significant relationships between posttest attitude toward 
CMI and course performance. Mathis, Sitilth, and Hansen (1970) measured 
attitudes toward Chi both before and after the instruction was presented. 
Errors per question attempted during the course of the instruction corre- 
lated negatively with posttest attitude but not with pretest attitude, 
Reid, Palmer, Whitlock, and Jones (1973) tneasured attitudes toward CAl 
before and after the instruction was presented* Pretest attitude^ post- 
test attitude, and attitude change were unrelated to lesson achievement 
and to the amount of time required to complete the lesson for the total 
sample* Although significant correlations between attitudes and perfor- 
mance were obtained for selected subgroups, it appears that no more than 
5.2% of the correlations aalculated were significant at the .05 level; 
thus these correlations could be attributed to chance alone. 

The above studies were conducted using college or graduate students, 
and the attitude 'scales used were not felt to be appropriate for use with 
enlisted military personnel. Consequently, Knerr and Nawrocki (1978) 
developed two attitude^toward^CAl scales, a 13-item pretest and a 37-item 
posttest, for use with military personnel, mien used in the evaluation 
of the PLATO IV CAI system at the U.S. Aiutiy Ordnance Center and School/ 
neither the pretest nor the posttest was correlated with the time required 
to complete a series of four lessons presented by CAI, the only perfor- 
mance criterion available. 
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The purpose of this study was to aontiriiie tliQ investigation of the 
relationships between military student attitudes toward CAI and student 
performance begun by Knerr and Nawrocki (1978), using a variety of maa- 
sures of performance in the instructional situation. 



METHOD 

Subjects 

SufajectB were 90 enlisted milita^ry personnel receiving training at 
the U.S. Mmy Enginear School during the period Septentoer to November 
1975. A description of salient student background information is pro- 
vided in Table 1. Median values show that the typical student was male, 
was in pay grade was 19 years old, had been in the military for 3 

months, had L2 years of formal education (all were at least high school 
graduates) I and could not type. 



Table 1 

Student Background Information 



Itain 



Pay grade 
Age 

Months in service 
Years education 
Typing speed 

(TOrds per minute) 



Male 



Median Range 



E-2 El - E5 

19 17 - 32 

3 2-53 

12 12 - 16 

0 0 - 100 



Female 



81 7 



Note. N ^ 88. Background data on two students were destroyed by 
systems failura. 
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^ttj^tude Scales 

^frie tw^ attitude scsl^s^ pretest andposttestf developed by lOierr arid 
WawrccKi (1^78) yrmm usafi, The pretest consists of L3 f ive-alternativa 
rtiialtlpZi choice items ^ ^sii previDuslj' administered^ it v^rai found. to ha\f^ 
an iiitemal c&riiistency CKuder^IlichardSDri Foritiula S; M 8) of .91*^ The 
postte^t cor^sists of 3 7 ttsms and has an internal corisis teracy of .B9p it 
can be di%'i4ed into two subtests ^ TOa first, the repeated sij^teBt ^ con-- 
sists of those 13 i tarns in the pretest (rephrased in the past tens€ where 
neceaiar/) gund ha^ an inCerjiaL consiitency of *85, The mLque s^test 
conpiiti of 24 additiaHaj^ iteins and has an interml aonsistency of ,81. 
Wne pretest and pesttest items are shov^n in appeniixea A and 3 
respectiv^ely^ . 

Tine scales had privi^ously been adriiinis tired in pmprnx and pencil toi^ 
mat, Prior to this st'Ud/' ttiey were modified for on-'line acainiiiistratton 
and scoring j^y the PLMO IV Ck3 system^^ ^he e^pej^iiaent^al prcceduja was 
such ttiab the scales h^d to be administered on an iiiddvi4u^l iasis . Com- 
puterised ad^nlnis'tr^ticn^ tt\Bti , offered a substantial tijiie sa^ir^gs for 
the axperimejitar, Xt al^o grDvicled the capability fox a%it&matic scoring 
and data rediiction. 

Pigti.re 1 shov^i a satfiple itein as it appeared initially to time students 
ttie student ^asponded hy entering the latter of him ox her cheici (a 
througta ^) via the teriftlml keyboard. This choice vas tfiart marteed by an 
aateriak, m4 the student was gi\ren an opfortuntty to change th^ ariswar 
before advaiicing to th^ r^eKt ^uestion^ as shown in Figure 2, Thse stiidant 
was required to arisv^et each queefcion in oxder to advance to the na^tt. 



This measure entaiis fe\/«r assumptioiis tlnan ths mors cDOTioaily ixsad Kuder-^ 
Ri chard son forniula 2 0 (KR- 20) . Spacif icalLy , m 8 assumes that the intern- 
correlation mtri^ has a rank of one, aat is, tliat the scale measures 
only ori€ factor, while KR= 20 a^siuiiiues^ in addition^ that all iteirL Imtercor-- 
relations and standard de^i&tions are equal. If these additiona 1 sssump-^ 
tiona are mst:, KR 8 and !C-R 20 will produce identical reLlability astimatia, 
If the/ aw riot met, 8 wi.ll prodoca higher and inore accurate reliability 
aatiinates than KR 20 , 

2 . . ^ . 

Commercial designations used only for preciaiori of description. 

Th€tr asa does not ccnstttute endorseiriant by tha Afitiy or thm Rma Besfiaren 
Instituita ~ 
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Most CDUtsag could toe taa^ht mora b feectiv*«ly by 



SL* Str&ngl^^ agree 
to. Agree 



rig^ure 1. pretest iteni 1, as it appears to th*^ student 
toefot€ M resporiBe ig selected* 



Most CDUrses couLd b>e taught more a ffectiv-*€ly by 

StrDngl.^ agree 
* to. Agree 

Undecified 
d. Disagree 

Yow ansvar is marked an ^ou caai change 

it by p^ssaing -BACK-, or continue -v^lttn th*€ next 
qu^stiDri by pressing -NEKT-^ 



rigure 2- fftatest item 1, as It appears to th^ studejit 
after a response Is seL^cCei * 
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The Lesson matarial cDnsistGd of a fQur^section lesson entitled "The 
El#Mntary Physiology of Audition (or How Your Ear Works) The sections 
dealt v^lth soundwaves, anatpmy of the ear, balance, and ear damage and 
degeneration, respectively. Following each section, the studants recaived 
Q series of review questions covering the objectives of that sectiori. 
Students who answered less than 85% of the review questions for any sec-- 
tian correctly w^ere required to repeat that section. After the fourth 
review vas saccessfully completed, the student was adininisterad a 6S-itein 
test Covering the objectives of the entire lesson. 



Pe^f ormance Criteria 



The following perforinance criteria were obtained for each stadanti 
(a) the amount of time required to complete the four lessoii and review 
sections (time); (b) the total score on the final end-of- lesson test 
(total scora) ; (c) the nuittoer of incorrect responses niade during the 
lesson and review sections (errors) ; (d) the percentage of .the total 
responses made during the lesson and reviw sections that v^era incorraet 
(percent errDrs) ; and (e) the number of reviews failed (re^^iew failures) . 
percent errors requires some additional explanation- It is the percentage 
of responses to which the student did not^ receive positive feedlDack (OK^ 
correct, right, etc). It is also similar to errors per question at« 
tempted. \^hich Mathis et al . (1970) found to be correlated with posttast 
attitude. - 



EKgerimgntal _P^rocedure 

Upon arrival at the experimental area, each student was given a short 
written explanation of the eKperimental procedures, After any questions 
were answered he or she was signed onto the PLATO IV system. The PLATO 
IV system controlled all student routing, lesson adjministfation, and data 
collection* Unless a problem arose, the student did not interact vrith the 
eKperimenter again until the end of the session. 

After being signed onto the system^ each student received a short 
lesson on the use of the keyboard and ternrtinal. This was followed by an 
on-line biographic questionnaire , the attitude pretest, the human audition 
lepson, the end-of -lesson test# and the attitude posttegt * 



It is not clear whether Kathis et al. (1970) used errors per at tempt or 
armors per question as their performance measure. In CAI leBSons, in 
general, a student may attempt questions more than once; the two measures, 
therefore, are not necessarily equal. Percent errors is equal to errors 
pmt attempt^ and a monotonic (but nonlinear) function of irrors per 
ques tion . 



Selection and Classification Test ScQges 



After the experiment had been completed, scores on the Arined Forces 
Qualification Teit (AFQT) and the General Technical (GT) aptitude area of 
the Army Classification Battery ware obtained for approxiinately one-half 
of the studants. These scores permitted examination of the relationships 
between "ability'* and attitude. 



RESULTS 

The results are organized around five separate issues* The first 
two are concerned with the characteristiGS of the attitude scales them- 
selves : scale administratiQn time and internal coniistenGy. The second 
two deal with the relationships between the attitude scale scores and thi 
perforinance criteria, and the relationships between attitude change and 
the performance criteria, respectively. The final issue is the relation 
ships among student ability, student performance, and student attitude. 



attitud e Scale AdministratiQn Time 

Although the amount of time required for the students to complete 
the attitude scales is not directly related to the primary objective of 
this research, it does provide sorne information about the feasibility of 
on-line administration. EKtremely long administration times would ba 
indicative of student problems and would also suggest that administration 
in an operational setting is not feasible. Table 2 shovrs the administra- 
tion time data for 8? students, {A systems error caused the loss of time 
data for the other three students 0 Mean administration time for the pre^ 
test is 4,90 minutes, with 90% of the students completing it in 6.90 ddn- 
utes or less. Mean adminiistratlon time for the posttesr Is 9.17 minutes^ 
mth 90% completing it in 11,80 minutes or less* 



Table 2 

Attitude Scale Administration Time (in Minutes) 



Pretest Posttest 



Mean 
Median 

Standard deviation 



Mote, N - 87. Data on three students were lost. 

7 



4.90 
4 .40 
1.81 



9,17 
8. 75 
2. 32 
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Interna l ConslsteTicy 



The item stain^ mean, standard deviation, and item- total correlation 
for each item are included in app^ndiKes A (pretest) and B (posttegt) . 
Table 3 presents a comparison of the internal consistencies (KR 8) of the 
on-line versions of the pretest^ total posttest, and unique and repeated 
siibtests with the internal consistencies of the previoiiSly adminiitered 
off-line (paper and pencil) versions. The internal consistencies for the 
on-line versions are .88 for the pretest, .93 for the repeated posttest, 
.87 for the unique posttest, and .93 for the total posttest. Each of 
these values was cotnpared with the internal consistency of the same scale 
when administered in paper and pencil format* Differences were tested, 
using the z test for the significance of the difference between two corre- 
lations. Table 3 shows the results of these tests. Wiereas it initially 
appeared that the on-line versions of the posttest had higher internal 
consistencies than did the off-line versions r this is supported only for 
the repeated posttest, for which the on-line version has an internal con- 
sistency of .93, and the off-line versiL.n, .85 (z ^ -2, AO, p < .01). 

Table 3 



h Comparison of the internal Consistencies (KR 8) 
of the On-^Line and Off-line Versions of the Attitude Scales 



Tests 


Off line 


. a 
On line 


dif fertince 


Pretest 




,88 


1*20 


Hepeated posttest 


C 

.85 


.93 


-2.40* 


Unique pQStt?;3Bt 


.81° 


.87 


-1,20 


Total posttest 


.89= 


.93 


-1,44 



b 

M s 228. 

% ^ 64. 

*p < .01. 
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Attitude-Performance Relationships 



The matrix of intercorrelations anrang the attitude scale scores arxd 
tiie lesson performanca criteria is shown in fable 4, The left mim of 
the table indicates that the performance criteria are highly correlated, 
with the absolute value of the correlations ranging from ,57 to .90, 
High values of total score reflect good performance, whereas high values 
of the other performance criteria reflect poor performance. Wius the 
correlations between total score and the other performance criteria are 
negative. 

The lower right corner of the intercorrelation tnatriic show© the 
intercorralations among the various attitude scales. Carralatioris of the 
pretest with the repeated, unique, and total posttests are ,78^ .59, and 
.71, respectively. The correlation between the unique and repeated post-' 
tests is .78. 

The attitude pretest appears to have no predictive validity. Corre-^ 
lations of the pretest with the performance criteria range from -.02 to 
.09. None of these correlations is significant at the *05 level. 

Two of the performance criteria show significant relatiohsliips with 
the postlesson attitude Scales. Total score is correlated with the uniqtia 
posttest (r ^ .21, p < ,05). Percent errors is correlated with tha 
wique posttest (r - -.22, p < .05) and the total posttest Cr ^ -.22. 
p <\05), The square of these correlations indicates that between 4.41% 
and 4.84% of the variance in posttest attitude is accounted for by the 
performance criteria. 



Attitude Change and performance 

The relationships between attitude change and performance ci^iteria 
%^ere also determined. The measure of attitude change used was residual 
change, that is, the portion of the posttest score that is independent 
of the pretest score. Use of residual change in this situation nas two 
advantaaes over the use of a raw gain score (posttest mnus pretest) t 
(a) it is not correlated with pretest score, while raw gain is negatively 
correlated with pretest scores and (b) it does not require that the pre- 
test and posttest scores have the same metric (Hanning B DuBoia, 1962). 
Correlations between the performance criteria and residual chang# were 
calculated by using the part correlation (McNeinar, 1962). Thus the corre- 
lation calculated is that between a performance criterion and a posttest 
score with the effects of the pretest removed from the posttest ecore. 
Three different residual changes calculated were the followingi pretest- 
unique posttest, pretest-repeated posttest, and pretest^total posttest. 
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Tibli 4 

Corrilations imm thi ftttitude Bmlm and tha Lesson Perfo™nc€ Critsrii 



Total Pircent Mvim Ripiated Uniqui 

score Errors errors Time failuris Pretest posttsit postta^t posttest 



Totil score LOO -.62** -ig** -.64** -.63^* -^.02 .12 Jl* .18 

Errotg LOO JO" .SS** ,79** -.00 -.12 -Al -43 

Percent errors l.OO .51** .56** -,06 -J9 -,22* 

Time 1,00 .76** -.02 -.06 -.12 -40 

Review failures 1.00 .09 ".02 -43 -.08 

Pretest LOO J8** .59** .71** 

tepsated posttest LOO J8** ,94** 

Unique posttest LOO .95** 

Total posttest LOO 

Maxiiuni N 90 S? 87 90 89 90 90 90 90 

Mean 79.31 5L84 30.72 106,79 'L 53 45.91 47.42 iLi6 139JB 

Standard deviation 9. 44 5LB 9,6S 34,57 L74 8.34 10. 2S 11,86 20,85 



■ .. . *p < .05, 
19 **p < .01. 
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The resulting correlations are presented in Table 5. Time and 
review failures are not correlated iignif icantly with any of the resid- 
uals* Total score and percent errors are correlated with each of the 
residuals. Errors are correlated only with the pretotal residual. The 
absolute value of these gignificant correlations ranges from .21 (errors 
arid the pretotal residual) to ,28 (total score and the pretotal and pre-- 
unique residuals) . Thus the proportion of variance in attitude change 
accountad for by the performance criteria, when significant, varies hB- 
twaen 4.41% and 7,84%, 



Table 5 

Correlationa Between Residual Attitude Change 
and the Lesson Perforniance Criteria 



Type of residual 



criteria 


N 


Pretotal 


Prerepeated 


Preunique 


Total score 


90 


.28** 


.22* 


.28** 


Errors 


87 


».21* 


-.18 


-,20 


Percent errors 


87 


^,25* 


-.22* 


-.23* 


Time 


90 


-.12 


-.08 


-.13 


Review failures 


89 


-.03 


.07 


-.09 



*p < 
**p < 



,05. 
.01. 



Student ^ility, Attitude^ and Per formance 

GT and MQT scores were available for 42 students. TOese scores 
were corralated with the performance criteria and the attitude scale 
scores, with the results shown in Table 6, Because the GT and AFQT scores 
are highly correlated (r (41) * .88, p <' ,01), thay show similar correla- 
tions with the other variables. Both are correlated significantly with 
each of the lesson performance criteria. In all cases, students with 
higher AFQT and GT scores obtained better scores on the performance 
criteria. 
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Table 6 



Correlations of GT and AFQT Scores With the Attitude 
Scale Scores and the Lesson Performance Criteria 



Maasure 




AFQT 



Total score 
Errors 

Percent errors 
Time 

Review failures 
Pretest 

Repeated posttest 
Unique posttest 
Total posttest 

N ^ 42 

Mean 

Standard deviation 



*p < .01, 

In contrast, GT and AFQT scores are not correlated with any of the 
attitude scale scores. Also, as shown in Table 7, GT and hFQT scores are 
not correlated with attitude change. In suminary/ total scora, errors ^ 
and percent errors are related to attitude change; GT and MQT scores are 
related to total score, errors, and percent errors i but GT and KFQT 
scores are not relatad to attitude change. A possible explanation for 
this is that the relationships between attitude change and performance 
are too small to contribute to relationships between "ability" and atti- 
tude change . 



,66* 

-.40* 
^.77* 
-.65* 
-.20 
--.16 
.04 
-.06 



111.79 
12.60 



.67* 
--.54* 
-^.42* 
79* 
^.66* 
^.06 
-.02 
.16 
.08 



66.14 
18.20 
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Table 7 

correlations Between Residual Attitude Change and GT and AFQT Scores 



Test Pretotal Prerepeated Preunique 

GT .10 



AFQT . 17 

N = 42 



-.02 .17 
+.04 .23 



DrSCOSSION 



Based on the results of this study, it appaars feasible to measure 
student attitudes toward CAI through the use of on-lina attitude scales, 
certainly thare are many questions that could be raised about such an 
approach. One such question is v^hether the mathod o£ administration (on- 
line versus paper and pencil) produces differential student responses. A 
related question is whether the method of administration has any effect 
on the metric properties of the scales. There was one significant dif= 
ference between the internal consistencieB of the on-line versions and 
those of Knerr and Nawrocki's (1978) paper-and-pencil versions,, however, 
the background characteriatias of the students in the two studies were 
not exactly equivalent and, more importantly, the amount of exposure that 
the students had to CAI, both before the administration of the Pretest 
and between the administration of the protest and the posttest, differed 
for the two groups. Thus the difference obtained may be a result of fac- 
tors other than the method of administration. 

Despite these uncertainties, on-line adminiatration appears to have 
certain adyantages. The first advantage is the reduction of personnel 
time required for administration and scoring; a second Is that the scales 
can be integrated into the computer system's management function, thus 
insuring that each student is administered the proper scale at the proper 
time- finally, the method of presentation eliminatea certain types of 
student errors, such as answering a question in the wrong space on an 
answer sheet, or failing to answer a question. 
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The internal consistencies of ,88 for the pretest and ,93 for the 
total posttest compare favorably with those of previously developed 
scales. 4 Few other scales that could be used as pretests have reported 
reliabilities. Gallagher (1970) reported an internal consistency of .937 
(Hoyt method) for the Michigan State University Learning Service "Mtituda 
Toward Learning Inventory,*' This scale is suitahle for use as a pretest, 
but it is not designed specifically to ineasure attitudes toward CM. 
Mathis et al* (1970) developed a 30^iteiti pretest but reported no internal 
consistency for it. 

Internal consistencies (KR 20 or equivalent) for other scales used 
as posttests are .89 for Brown*s (1966) 40-^item scale and ,82 for the 27- 
item scale developed by Mathis et al. (1970). Many scales, such as those 
used by Doty and Doty (1964) and Reid et al. (1973) ^ have no reported 
internal consistencies* 

Evidence is mounting that preinstructional attitude toward CM does 
not affect subsequent performance in the instructional situation, at 
least when relatively naive students are involved. Knerr and Mawrocki^ 
(1978), using time as the criterion, found this to be the case. Mathis 
et al. (1970) obtained similar results, using errors per question 
attempted as the criterion. In the present study, no prediGtive relation- 
ships were found, although a variety of lesson performance arlteria were 
used. 

These results may not be generali^able to students who have had ex-- 
tensive previous experience with CAI . Mathis et al. (1970) and Knerr and 
Nawrocki (19 78) used as subjects students who had no previous experience 
with CAI . In the current study, the eKperience of the students prior to 
the administration of the pretest was limited to the time required to 
complete the terminal familiariEation lesson and the biographic infaritia'* 
tion questionnaire — less than 30 minuteD* Thus C^I was still a novel and 
somewhat unfamiliar situation for the students. It is not known whather 
similar results would have been obtained if the students had previously 
received instruction by CAI . 

Certain aspects of student performance in the instructional situation 
are related to posttest attitude (total score and percent errors) and to 
attitude change during the course of instruction (total score, errors, and 
percent errors). These effects, however, were not large. None of the 



"Other internal consistencies reported have been calculated using formula 
KR 20 or an algebraic equivalent. For the scales used in this study, KR 
20 internal consistencies were .87 for the pretest and .92 for the post= 
test. The KR 20 values are less accurate than the KR 8 values, but they 
provide a fairer comparison with other scales. 
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performance criteria accounted for more than 5% of the variance in post- 
teat attitude or imxm than 8% of the variance in attitude change. 

Student ability, as weasurad by APQT and GT scores, was relatea to 
neither attitude scale scoraB nor attituda change. This result provides 
some evidence for the construct validity of the attitude scalea. At a 
mininiuin, it shows that the attitude scales rnaaaure something other than 
student ability. This is a neoassary requiratnent for the establishment 
of construct validity, but it is not a sufficient one. According to the 
APA Standard s for E ducation al and Psyeholo gleal Tests , 

Evidence of construot validity is not found in a 
single study; rather, judgments of construct 
validity are based upon an accumulation of research 
results (1D74, p. 30). 

Future research examining the relationships between the attitude scale 
scores and other variables would be required to establish the construot 
validity of the scales. 



CONCLUSIONS 



In view of the lack of predictive relationships between the attitude 
pretest and subgaquent parformance, measuririg pretest attituda for naive 
students would seem to be of little value unless ona is interested in 
studying attitude change, per se. Student attitudes do not appear to be 
useful for deciding whether a particular student should receive instruc- 
tion by CAI or by some other medimn. This nay not be the case for stu- 
dents who have had extensive previous experience with CAI, however. 

Preparation of CAI lesson materiala that produce low student error 
rates during the course of instruction and on the postlesson test ma^ be 
beneficial in terms of maintaining or produoing positive student atti- 
tudes toward CAI. It has been demonstrated that certain aspects of stu- 
dent performance are related to subsequent student attitudes? neverthe- 
less, a causal relationship has yet to be astablished. 
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AIM HNDIX A 
PRETEST ITEM SUMMARY 



This appendix inulud^^ d list of all praU^sL ILems, with the mean, 
standard deviation (s.dj, and item-total correlation (r^^) for each. 
All odd-nunibered items are negatively worded, that is, a "strongly agree 
"all of it," "all the time," or "quite often" response is the one least 
favorable toward CAI . All even-numbered items are positively worded. 

Response alternatives are coded as follows i 



A - a. Strongly agree 

b. Agree 

c. Undecided 

d. Disagree 

e. Strongly disagree 



P - a. All of it 

b. 75% 

c. 50% 

d. 25% 

e. None of it 



T - a. All the time 

b. Most of the time 

c. Some of the time 
d- Only occasionally 
e * Never 

F ' a. Quite often 

b. Often 

c. Occasionally 

d. Seldom 

e* Very seldom 

All data are based on a sample si^e of 90. 
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Item 

no , 



Item stem Mean s.d. r^^ 



1, Most courses could be tauqht more 
effeGtivaly by a regular teacher 

than by computer. (A) 2.96 0,92 ,47 

2, I would like to take a lesson 

taught by CAI. (A) 4.21 0.91 ,60 

3, Taking a lesson taught by CAi 

would make me nervous* (A) 3.09 0.98 -37 

4, Taking a course taught by coniputer 
would be more interesting than 
taking the same course taught in 

some other way. (A) 3,60 1,14 ,66 

5, People should be taught by other 

people, not by machines* (A) 3*13 1.05 .62 

6, I would prefer to have most courses 
taught by computer rather than by 

other teaching methods. (A) 2,97 1,09 .74 

7* I think I would feel isolated and 
alone while taking a course taught 

by computer. (A) 3,51 1,13 ^67 

8, X th^ . I would feel challenged to 
do my best work while taking a 

course taught by computer. (T) 3,81 1*04 ,62 

9, It would be boring to take a course 

taught by computer, (A) 3.53 1.10 *64 

10. I think it would be easy to under-- 
stand the material in a course 

taught by computer, (A) 3.49 0.81 .54 

11. Students are being treated more 

and more like IBM cards. (A) 2.99 1.34 ,61 

12. How much of this lesson do you 
think you would like to have 

taught by computer? (P) 4.28 1.08 .62 

13* Taking a lesson taught by computer 

would be too mechanical. (A) 3*54 1*06 .77 
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APPENDIX B 
POSTTEST ITEM BtlMMAHY 



This appendiK includes a list of all posttest items, with the mea 
standard deviation (s.d.), and item-total correlation {r^^ for each,^^ 
odd-numbered items are negatively worded, that is, a "strongly agree," 
"all of it/' "all the time/' or "quite often" response is the one leas 
favorable toward CAI . All even-nuntoerod items are positively worded. 

Response alternatives are coded as follows: 



A - a* Strongly agree 

b. Agree 

c- Undacided 

d* Disagree 

e. Strongly disagree 



P - 



d. 



All of it 

75% 

50% 

25% . 

None of it 



T = a- All the time 

b. Most of the time 

c. Some of the time 

d. Only occasionally 

e. Never 



F - a. 

b. 
c 

d. 
e ^ 

All data are 



Quite often 
Often 

Occasionally 
Seldom 
Very seldom 

based on a sample size of 90. 
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Item 
no* 



Item stem 



Mean s,d. 



1, The way the material was presented 
Uo me made me feel that no one 
really cared whether I learned or 

not. (A) 3.67 1. 17 ,55 

2, Based on my experience with this 
lesson^ I prefer Chi to other meth= 

ode of instruction. (A) 3.21 1.14 .66 

3. The method by which I was told 
whether I had given a right or 

wrong answer became boring, (A) 3.50 1,21 ,66 

4. I felt as if someone were engaged 

in conversation with me. (T) 2,91 1.25 .51 

5* I was concerned that I might not 

be understanding the material, (T) 3.53 0.93 ,25 

6, The responses to my answers were 

appropriate, (T) 4.11 0.81 .35 

7. I felt uncertain as to my perfor^ 
mance compared to the performance 

of others. (T) 3.34 1.28 ,27 

8. I knew whether my answers were 
correct or not before I was told. 

(F) 3.50 1.26 ^,01 

9, 1 found myself just trying to get 
through the lesson rather than 

trying to learn. (T) 3.62 1.04 .63 

10, I was encouraged by the responses 

given to my anst^ers. (T) 3.61 1,01 .63 

11, 1 guessed at the answers to 

questions, (F) 4.39 0.83 ,35 

12, I was able to work at my own 

pace. (T) 4.80 0.62 .23 

13, In view of the time allowed, I 
felt too much material was pre^ 

sented. (T) " 4.19 1.10 ,39 
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14, 1 felt as if I had a private 
instructor, (T) 

15, I was more involved in running the 
machine than in understanding Uhu 
material. (T) 

16, I was aware of efforts to suit the 
material specifically to me. (F) 

17 , I found it difficult to concen^ 
trata on the course material because 
of the machine. (T) 

18* Compute r^assis ted instruction made 
it possible for me to learn 
quickly, (A) 

19. Questions were asked which were 
not relevant to the material 
presented. (T) 

20. The responses to my answers seemed 
to take into account the difficulty 
of the question* (T) 

21. In order to get mora information 
from the machine^ I gave answers 
which I knew were wrong. (F) 

22. In view of the effor^t I put into it^ 
I was satisfied with what I learned 
while taking CAI . (A) 

23 ^ ,1 was given answers but still did 
not understand the questions. (F) 

24* In view of the amount I learned, I 
would say that CAI is superior to 
other teaching methods. ih) 

25 , Most courses could be taught more 
effectively by a regular teacher 
than by computer. (A) 

26 , I would like to take another lesson 
which uses CAl. (A) 

27, Taking a lesson taught by CAI made 
me nervous* (A) 



3,77 1.40 .53 

4.11 0.95 .52 
3,24 1. 33 .49 

4.24 0.95 -50 

3.92 1.05 .64 

4.67 0.67 .11 

3-04 1-23 .28 

4.51 0.92 .29 

4.34 0.85 .65 

4.17 0.96 .60 

3.44 1.06 .64 

3.12 1-22 .67 

4.13 1.10 .73 
4.10 1.08 .34 
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28. The material taught by coniputer 
was more interesting than taking 
similar material taught in some 

other way. (A) 3,64 1.2.1 .62 

29. People should be taught by other 

people, not by machines. (A) 3*29 1.14 .71 

30. I would prefer to have most courses 
taught by computer rather than by 

other teaching methods. (A) 3.19 1.16 .72 

31* I felt isolated and alone while 

working with the computer. (A) 3.47 1.22 .58 

32- I felt challenged to do my best 
work while being taught by 

computer. (T) 4.03 1.00 .68 

33. It was boring to learn material 

taught by computer. (A) 3 = 74 1.21 * 72 

34* It was easy to understand the 

material taught by computer. (A) 3.92 0.89 .59 

35. Students are being treated niore and 

more like IBM cards. (A) 3.21 1.28 .58 

36. How much of the lesson you just 
completed do you think should be 

taught by computer? (P) 3.90 1.26 .71 

37. The lessons taught by computer 

were too mechanical. (A) 3.67 1.13 .75 
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